[intro music]
Host – Dan Keller
Hello, and welcome to Episode Ninety-eight of Multiple Sclerosis Discovery, the podcast of the MS Discovery Forum. I’m Dan Keller.
Today's interview again features Dr. David Baker, Professor of Neuroimmunology at Queen Mary University of London in the U.K. We spoke at the ECTRIMS conference last fall. In part one of our interview he raised the issue of why there has been very poor translation from animal models to clinical trials. Today, Dr. Baker, also known as the ”Mouse Doctor” for his work with animal models, lays out why this situation exists and what to do about it.
Interviewee – David Baker
I think there’s many reasons why, and I think we all have our failings. And one can point the finger at the animal models, which a lot of the clinicians do, saying it’s the animal model’s fault, which is possible. But I think also we have to look at humans and how humans use their animal models. And then how humans translate the data from the animal models into the clinic, because I think there’s many failings along the line, and I think that’s one of the reasons for the failing between the two.
I think one of the failings is, in terms of the animal models, that when we do our animal models for these, we’re looking for mechanisms not treatments. And so about 70% of studies give drug before disease is ever induced, which never happens in a human. You know, you go after you’ve had one or two or more attacks before you’re given drugs. We also use the drugs in a way that are never used in a human, so people will do what they call a prophylactic drug where they’ll give it before the disease manifests itself. Or a therapeutic dose, which is probably when the animals are showing their symptoms. But in reality, a human would be getting steroids at that time point. They would never get a DMT. So you’re not comparing, you know, apples with apples. You’re comparing apples with pears, and I think that’s one of the problems.
And I think, you know, if you try and block an immune response from being generated, that’s quite easy compared to stopping an immune response once it’s been generated, because immunity’s about giving life-long protection against infections. And so I think it’s a different type of beast to target. So I think this is where the animal models could do it, because EAE is one of the few where you have this relapsing-remitting disease course. But it’s very, very rare that people actually start to treat in between attacks to block further relapses. I think that’s one of the problems.
The other big problem is the dose; the dose relationship between animals and humans. There’s a tendency we just keep giving more and more and more and more, and eventually the drugs will work. But you’ve got this problem that animals are very liable to be stressed, and we call it the building site effect, so construction site effect. And if you have lots of loud noises, it scares animals. They get very stressed, and your EAE just disappears. And likewise, if you just give lots and lots of drug, that probably tastes nasty. They get stressed out as well. And I think many of the so-called wonder cures – cures of the week – are because we’re just giving too much, which doesn’t have a relationship to what the human dose is going to be.
And then, likewise, I think we’ve got too much of a publication bias for the need to generate positive data. And I think what we then have to do is we have to look at the quality of the data. And I think there has been a lot of failure to replicate data. I think some of that is because some studies lack quality control, and the way I look at that – and I could be wrong; obviously it’s an opinion – but if you look at the way that EAE is scored (it’s normally a scoring system 1 to 3 or 1 to 4) and then you have your drug, which may be, you know, takes your control down from 3 down to a 1. But then, every now again, you look at the studies where it goes either way, and your controls are at 1 and it goes up to 3, and I ask the question how do you get a score of 1? Because if you had four animals, they’re all scoring 1. Or is it three animals score 0 and one score 4, and that will give you a score of 1. And I think if people were made to actually put the data about how many animals got disease, we’d be able to interpret those line graphs. Because I feel that, in many cases, some of those graphs lack quality control.
If you have a robust quality control system, your control group should be giving you roughly the same type of scores every time. But in individual papers you can see, in some groups you have a score of 1 in the control group. The next experiment it’s a score of 3. To my mind I think if you look at that, then those are probably the experiments are much more likely not to replicate. So I think you have to be, obviously, skeptical, but I really would like people to actually probably give us the information about how many animals got disease – what is their mean score – in addition to those line graphs. Because without that, they’re impossible to interpret.
So that’s, you know, kind of one problem of the animals. And then for the humans, you have the same problems. So they over-interpret the animal data. The people doing the clinical trials are very, very rarely the people who came up with the idea. So if there’s a weird side effect that you may know about, you know, that’s not translated to the person who’s actually doing the study, because they don’t talk to the basic scientists. Then they probably underpower the studies. They don’t necessarily pick the right outcome measurements. So I think there’s many failings in both sides of the equation, and it’s not always the animal model. But I think unless we kind of up our game, I think it’s going to be very difficult for the people who are working on animal models, because you know, there are treatments that come along for, you know, the immune part of multiple sclerosis.
And if you’re thinking about the ethical use of animals, it’s much harder to make the ethical argument that you should be using disease models which are very severe for the animals to try and work out fundamental parts of biology. And, therefore, I think we’ll find that you know the funding agencies start to say, well, why are we funding this work? So I think we need to have good quality work, because if we don’t have good quality work, it allows that clinical view that animal work doesn’t really deliver the treatments. And I think they can deliver the treatments, but we just have to use our animal studies wisely to ask questions rather than, you know, blindly saying this will work in multiple sclerosis because it works in EAE. That doesn’t make sense to me.
Interviewer – Dan Keller
Do you have any succinct tips for people who are either reviewing papers on animal studies or people who are reading those papers once they’re published or even the general public reading a news story?
Dr. Baker
Well my first tip would be probably – and this is okay as an opinion – but, you know, EAE data is nonparametric. It goes 1, 2, 3, 4; it’s not a continuous scale, so first tip is don’t use, you know, the t-test of parametric data on nonparametric data. And that does make a difference. There is a Nature paper published this year that was analyzed with a t-test. If you analyze it with a Mann-Whitney test, which you should have done, the data becomes nonsignificant. So rather than the take home message is, you know, this is a new wonder drug for multiple sclerosis, their answer should have been you have to go back and reproduce your EAE experiment because it didn’t work. So I think that would be the first tip. And then the second tip, I would really like people to say, tell us how many animals get disease and on what level and when, so we can interpret the line graph.
MSDF
This is something that you routinely see in oncology done right. They talk about percent of responders, and among responders, what was the shrinkage of the tumor? They don’t average it out among all the people who dilute it out by not responding.
Dr. Baker
Well I think one of the problems as well is we’ve also got this publication bias. We’ve got you know this urge to see positive data, and I think that skews the whole system.
MSDF
Has anything changed since you came out with a response to the animal checklist?
Dr. Baker
I think, sadly, no, but we’re actually doing the checklist again, so we will be able to see if things have changed. I don’t think it has. I think the message hasn’t gotten through. But I think – this is, again, another one of those nails in the animal model coffin that, if we don’t up our game, we’ll be seen to be doing an inferior quality work and eventually we’ll get discarded. So I know that some of the grant councils are, as you know, saying this is a condition of your grant. But I think you know it’s been slow to change, and I think one of the reasons is actually people who are leaders of the field actually are some of the people who are some of the worst offenders. So we’re leading by bad example rather than good example.
MSDF
We don’t want to leave the listener with the impression that you’re against animal models. I mean, you’re known as “Dr. Mouse,” so you know I guess you just want to see them done well.
Dr. Baker
Yes, I’m passionate. I mean, I really you know believe animal models have a real positive impact to do. And I’ve been really lucky in the recent years is that, you know, some of those animal models – and work we’ve done from animal models – is going through into humans and you know is starting to make the difference. So you know our work with the Cannabis was great. You know, it shows that you know our animal work has validity. Without the animal model stuff we’ll never really understand the biology. You can’t do all the experiments in humans. You do need experimental systems to be able to ask questions. And you need to be able to invent.
And you know there is some fantastic work. You know I’ve picked up the papers, and I get really excited by it, but I think, at the same time, we have to also be a drum to say, you know, try and improve the quality. Because, at the end of the day, it’s more likely that if you’re doing good quality animal experiments, that other people will be able to replicate it. And it will move the field on further and faster. And I think if people believe what we produce as being good solid work, then it’s going to be a win-win situation.
MSDF
It would be nice to see sort of a meta-analysis of animal studies that are considered to have been done well versus those not and see which ones translated into advances in the human situation, because so many times they say, well, sure it works in animals, but it doesn’t work in humans. Well if it works in animals because it was set up not so well, then that might be a reason not to work in humans.
Dr. Baker
Yes. I think you know the problem of animal models has got nothing peculiar to the multiple sclerosis field. It’s just a common theme. And I think that tells me it’s not a problem of animal models, because if it’s so common in every other discipline, it tells us it’s something how we use the animals is the fundamental problem. Now, you know for MS, we don’t really know. I mean, I think this going to be the – we’re at ECTRIMS now, and I think the whole world can change a little bit today, or in the next few days, because we’ve always thought of MS as being a T cell-mediated disease. Now that may be still the true answer, but now we’re starting to see ocrelizumab, which is a big B cell depleting antibody probably – I’m predicting – to have as good an effect as anything that the T cell you know brigade has ever done. And, in fact, if you look at most of the MS drugs, you would say that most of them actually are inhibiting B cell function.
Now, does that tell us that B cells are driving the disease? It may well do. Or it may well not. Now some people could argue – and they will – you know they’re the reservoir for the virus that causes multiple sclerosis. And then other people will say, well, actually the antigen-presenting cells. And let’s see, but I think what we’ll find is you know EAE will have to have changed its focus. We’ve been focusing our studies on T cell biology, but in fact, the T cell-inhibitory molecules haven’t really delivered. So is that right? And it may well be you know we have to think of a different biology. But EAE can certainly do that if need be.
So we’ll have to you know try and work out how do these B cell-depleting agents work. Is it you know via antigen presentation or not? I don’t know.
MSDF
We’ve always thought of T cells as regulating B cells. Now it looks like they both regulate each other.
Dr. Baker
I mean, I have my history in skin diseases, and when I first started working, actually my boss was more interested in B-regulatory cells. T-regulatory cells kind of hadn’t really existed at that time point. So I think we’re trying to reinvent the wheel. If we look throughout the literature, it’s a cross-talk between T and B cells are probably the answer. And we’ll see. Again, from our animal studies, we’ve had animal studies where we’ve manipulated the immune system making sure that has a positive effect. We’ve been able to translate that, so we have an N of 1 where we’ve got rid of somebody’s neutralizing beta-interferon antibodies by antigen-specific mechanisms. Now if we could translate that into MS, then we may have a way of treating MS. But we’ll see.
MSDF
Very good, thank you. I appreciate it.
Dr. Baker
Okay.
[transition music]
MSDF
Thank you for listening to Episode Ninety-eight of Multiple Sclerosis Discovery. This episode is the final one in our series of MS podcasts. We hope that the series has been enlightening and has spurred further discussion about the causes of MS and related conditions, their pathological mechanisms, potential ways to intervene, and new research directions. We’ve tried to communicate this information in a way that builds bridges among different disciplines, with a goal of opening new routes toward significant clinical advances. Although we won’t be adding any new podcasts, the series will remain available on the MS Discovery website for the foreseeable future.
This podcast was produced by the MS Discovery Forum, MSDF, the premier source of independent news and information on MS research. Msdiscovery.org is part of the nonprofit Accelerated Cure Project for Multiple Sclerosis. Robert McBurney is our President and CEO, and Hollie Schmidt is Vice President of Scientific Operations.
[outro music]
We’re interested in your opinions. Please join the discussion on one of our online forums or send comments, criticisms, and suggestions to editor@msdiscovery.org.
For Multiple Sclerosis Discovery, I'm Dan Keller.
